Image-size dependent facial caricaturing

ABSTRACT

The present invention applies prior art findings with regards to increased recognisance of caricatured images by providing a method and system which provide for the level of caricaturing to be applied to an image to be set in dependence on the intended size of the caricature image. Preferably, the caricature level is set as a generally inverse function of the image size, such that the smaller the image the greater the level of the caricaturing that is applied. In such a case increased recognisance of subjects represented in smaller images may result.

TECHNICAL FIELD

The invention relates to a method and system for generating caricatureimages, wherein the level of caricature is dependent on the intendedsize of the resulting image.

BACKGROUND TO THE INVENTION AND PRIOR ART

Automatic caricaturing methods and systems are already known in the art.Brennan, S. E. in “Caricature Generator: The Dynamic Exaggeration ofFaces by Computer.” Leonardo, Vol. 18 no. 3, pp. 170-178 describes acomputational model of caricature which allowed a two dimensional linedrawn caricature to be generated from photographs. The user traces overthe original image (by placing a set of markers over the image) togenerate a veridical line drawing of the subject. An example of such anoriginal image and the resulting veridical line drawing are shown inFIG. 11. Here, an original image as shown in FIG. 1(a) results in averidical line drawing as shown in FIG. 1(b).

Having obtained the veridical line drawing of the subject, this drawingis then compared with a corresponding line drawing of a “mean” or“prototype” face, by which is meant an average face of a group usuallycomprising the same race, gender, and colour as the subject. Thus, forthe white Caucasian male shown in FIG. 11(a), usually a prototype faceof an “average” white Caucasian male would be used. In somecircumstances prototype faces from different ethnic groups may be used.

Rowland et al in Imagina 97—Conferences—ACTES/Proceedings, February,Monte Carlo, (1997), pp 159-175 describe how a prototype face may bederived as follows. A prototype can be defined as being a representationof the consistencies across a collection of faces. For example, aprototypical male Caucasian face would contain all that is consistentabout Caucasian faces and can be generated by calculating a mean facefrom a set of Caucasian faces.

To derive the prototypical shape for a group of faces, the delineationdata for each face are first “normalised”, making the faces nominally ofthe same size and orientation. The left and right pupil centres provideconvenient landmark points for this process. The first step is tocalculate the average left and right eye positions for the wholepopulation. The next step is to apply a uniform translation, scaling,and rotation to the (x, y) positions of all the feature points, thusnormalising each face to map the left eye to the average left eyeposition and the right eye to the average right eye position. Thisprocess maintains all the spatial relationships between the featureswithin each face but standardises face size and alignment. It is thenpossible to calculate the average positions of each remaining templatepoint (after alignment), the resulting data constituting the mean shapefor the given population. A line drawing of the resulting “mean” orprototype face can then be obtained. An example line drawing of a meanface is shown in FIG. 2.

Once a prototype has been formed for a collection of faces it ispossible to generate caricatures by accentuating the difference betweenan individual face and a relevant prototype. After normalising thefeature location data from the prototype to the eye positions of anexample face, all feature points on the example face can be shifted awayfrom their counterparts on the prototypical face by a given percentage.This percentage is the amount of caricature and can be thought of asextrapolating a morph between a prototype and the example face. If thepercentage is 100% then the product of the manipulation will be theprototype, if the percentage is 50% then the result will be halfwayalong the morph between the prototype and the example face, if thepercentage is 0% then the example face is returned, if it is −50% then acaricature of the original face is the result. More generally, anypercentage less than 0% will result in a caricatured face.

It has also been shown in Rhodes, G. & Brennan, S. E. (1987).Identification and Rating of Caricatures: Implications for MentalRepresentations of Faces. Cognitive Psychology, 19, 473-497 thatcaricaturing of faces results in greater recognition of the caricatureface as the subject than an un-caricatured face.

SUMMARY OF THE INVENTION

The present invention inventively applies the above findings of theprior art with regards to increased recognisance of caricatured imagesby providing a method and system which provide for the level ofcaricaturing to be applied to an image to be set in dependence on theintended size of the caricature image. Preferably, the caricature levelis set as a generally inverse function of the image size, such that thesmaller the image the greater the level of the caricaturing that isapplied. In such a case the inventors believe that increasedrecognisance of subjects represented in smaller images may result.

In view of the above, from a first aspect the invention provides amethod of generating a caricatured image, comprising the steps of:

determining a caricature level value in dependence on an intended sizeof the caricatured image to be generated; and

generating the caricatured image using the caricature level value thusdetermined.

The invention provides the advantage that enhanced recognition ofsubjects shown in images can be obtained through caricaturing, whereinthe enhanced recognition is dependent on the intended size of the image.

Preferably, the determining step further comprises determining thecaricature level value as a generally inverse function of the intendedsize of the caricatured image to be generated. This provides theadvantage that recognition of smaller images may be enhanced to agreater degree through increased caricaturing than larger images.

From a second aspect there is further provided a system for generatingcaricatured images, the system comprising processing means arranged inuse to:

i) determine a caricature level value in dependence on an intended sizeof the caricatured image to be generated; and

ii) generate the caricatured image using the caricature level value thusdetermined.

The invention in the second aspect provides the same advantages andfurther features as previously described in respect of the first aspect.

From a third aspect the present invention further provides a computerprogram or suite of computer programs arranged such that when executedby a computer system it/they cause the computer system to operateaccording to the aforementioned first aspect.

Moreover, from a fourth aspect, there is also provided a computerreadable storage medium arranged to store a computer program accordingto the third aspect of the invention. The computer readable storagemedium may be any magnetic, optical, magneto-optical, solid-state, orother storage medium capable of being read by a computer.

BRIEF DESCRIPTION OF THE DRAWINGS

Further features and advantages will become apparent from the followingdescription of embodiments of the present invention, presented by way ofexample only, and by reference to the accompanying drawings, whereinlike reference numerals refer to like parts, and wherein:

FIG. 1(a) is an example input image to an embodiment of the invention;

FIG. 1(b) is a veridical line drawing corresponding to the image of FIG.1(a);

FIG. 2 is a line drawing of a mean or “prototype” face used in theembodiments of the invention;

FIG. 3 is a block diagram showing the main elements of an embodiment ofthe invention;

FIG. 4 is a flow diagram setting out the processing steps performed inan embodiment of the invention;

FIG. 5 is a plot of an example function varying the caricature level independence on the image size used in an embodiment of the presentinvention; and

FIG. 6 are line drawings illustrating output images of various sizesfrom an embodiment of the invention.

DESCRIPTION OF THE EMBODIMENTS

Embodiments of the invention will now be described with respect to FIGS.3 to 6.

FIG. 3 illustrates the hardware which provides an operating environmentfor the embodiment of the present invention. Particularly, the operatingenvironment comprises a general purpose computer 1, which may be astandard desktop, laptop, or the like. Moreover, although we have shownthe general purpose computer 1 to be a desktop computer, it should alsobe understood that almost any device capable of performing processingoperations on data may be used, and in particular mobile devices such asmobile telephones, personal digital assistants, or the like. Within theembodiment to be described, however, let us assume that the processingdevice is a general purpose computer such as is commonly known in theart.

The general purpose computer 1 is provided with a storage device 20 inthe form of a hard disk drive, floppy disk drive, memory, opticalstorage, or the like, as is also commonly known in the art. Stored onthe storage device 20 are various programs and data 32 to 40. Moreparticularly, the embodiment of the invention provides a caricaturingprogram 34, which acts to generate caricatured images given an inputimage, as well as a control and interface program 36, which whenexecuted acts to control the caricaturing program, and to provide aninterface thereto. Additionally stored on the storage device 20 is datacorresponding to an input image 38, as well as data corresponding to areference image 40. Both the input image data 38 and the reference imagedata 40 are also required by the embodiment of the invention. Thestorage device 20 also stores another computer program, which, in thisexample, is a game program 32. The game program 32 will be used todescribe a second embodiment of the invention later.

Having described the operating environment required by the embodimentsof the invention, and the programs and data provided by the embodiments,the operation of a first embodiment will now be described with respectto FIG. 4. In the first embodiment of the invention to be described, theintended caricature image size is determined by a user using the controland interface program 36. That is, assume here that a user wishes toproduce a caricatured image which may then be saved and/or displayed onthe screen.

In order to achieve this object, the user first accesses the control andinterface program 36, for example by double clicking on a suitable iconon the general purpose computer 1 desktop. This action acts to launchthe control and interface program 36, which provides a graphical userinterface for the embodiment of the invention, and provides user controlbuttons and the like in order to allow the user to start thecaricaturing process, stop the caricaturing process, and to input imagesto be used as the basis for a caricatured image to be generated.Additionally, in the first embodiment the control and interface program314 provides further user controls to allow the user to set the size ofthe caricature image to be generated.

In view of the above, and referring to FIG. 4, at step 4.2 a user startsthe control and interface program 36, and uses the control and interfaceprogram 36 in order to specify an input image. The input image may be,for example, captured by a digital camera, or may be an already existinginput image which the user has downloaded or the like. The input imagemay be a photograph such as shown in FIG. 1(a), or alternatively may bea veridical line drawing such as is shown in FIG. 1(b). Whatever theformat of the input image, the control and interface program 36 acts tostore the input image as input image data 38 in the storage device 20.Additionally, where the input image is a photograph, and where requiredfor the particular caricaturing algorithm being used, the control andinterface program 36 acts to guide the user to generate a veridical linedrawing of the subject in the photograph, which may then be used as thebasis for caricaturing later on. In such a case the generated veridicalline drawing image is also stored as input image data 38 in the storagedevice 20.

Having received the input image and stored it in the storage device 20,the next step in the process at step 4.4 is that the control andinterface program 36 prompts the user to select an intended image sizefor the caricatured image to be generated. As mentioned previously, thecontrol and interface program 36 provides user controls as part of thegraphical user interface in order to allow for the user to set or selectthe intended caricature image size. Therefore, at step 4.4 the user setsthe intended caricature image size using these controls.

Having received the intended caricature image size from the user, atstep 4.6 the control and interface program 36 acts to set the caricaturelevel in dependence on the intended image size. It should be understoodthat the caricature level may be set as any function of the intendedimage size, but in the embodiment described here the caricature level isset as a generally inverse function of the intended image size, butpreferably only over a sub-range of the possible intended image sizes.An example function which may be used to set the caricature level isshown in FIG. 5.

FIG. 5 illustrates a plot of the caricature level against the intendedimage size provided by the present embodiment. Here it will be seen thatthe caricature level varies generally inversely with the intended imagesize above a threshold image size S1. The precise function which definesthe variation is shown here as a curve, but the parameters of the curvemay be varied to give shallower or steeper curves, as shown. Preferably,the curve tends to a caricature level of 1.0, as the image sizeincreases.

Additionally, the maximum caricature level is shown here as 2.0, but theinvention is not limited to this, and any caricature level may be chosenas the maximum caricature level. The inventors have found, however, thatcaricature levels above 2.0 result in lower quality results.

From FIG. 5 it will be seen that for image sizes less than the thresholdimage size S1, the caricature level is set to 1.0. A caricature level of1.0 means that the output caricature image will be identical to theinput image upon which the caricature image is to be based i.e. nocaricaturing will be applied. Moreover, note here that a caricaturinglevel of 0 means that the output caricature image will be the same asthe mean or prototype face as shown in FIG. 2.

The reason for not applying caricaturing to images smaller than thethreshold image size S1 is that for such small images the facialfeatures contained within the image would tend to be so small that theywould be indiscernible in any event. Therefore, for reasons ofcomputational efficiency, the caricaturing algorithm is not applied toimages smaller than the threshold image size S1, and instead the inputimage is output instead. In such a case the processing then ends.Example threshold levels for S1 could be an image size of less than20×20 pixels i.e. S1=400 square pixels. Other threshold sizes may ofcourse be chosen.

The caricature level setting function as shown in FIG. 5 may beimplemented by storing an appropriate mathematical function whichdefines the function, or alternatively by storing the function in theform of a lookup table of caricature level values against intended imagesizes.

Returning to FIG. 4, having set the caricature level and assuming theimage size is greater than the threshold size S1 such that thecaricature level is greater than 1, then at step 4.6 the control andinterface program 36 causes the general purpose computer 1 to run thecaricaturing program 34, and the caricature level which was set at step4.6 is passed to the caricaturing program 34 as input. Additionally, thecontrol and interface program 36 also passes the input image data 38, aswell as information relating to the intended size of the caricatureimage to be generated to the caricaturing program 34, as additionalinputs. Next, at step 4.8 the caricaturing program 34 acts to generate acaricature image based on these inputs passed thereto. It should benoted here that the algorithm used to generate the caricatured image bythe caricaturing program 34 may be any caricaturing algorithm whichallows a caricaturing level to be set. As a non-limiting example anysuch algorithm already known in the art may be used, but in particularthe Brennan algorithm referenced previously is particularly envisaged,and those features thereof necessary for fully understanding the presentinvention are hereby incorporated herein by reference.

Once the caricature image has been generated, at step 4.10 thecaricaturing program outputs the generated image, and in the firstembodiment the control and interface program 36 displays the generatedimage to the user using the graphical user interface provided thereby.

Therefore, in the first embodiment the user is asked to set the intendedimage size, from which the caricature level is then set according to agenerally inverse function such as is shown in FIG. 5, and a caricaturedimage then produced using the set caricature level, by a knowncaricaturing algorithm. The generated caricature image can then bedisplayed to the user.

FIG. 6 illustrates the effect of the intended caricature image size onthe output of the present invention. Particularly, FIG. 6 illustratesfive output caricature images, wherein the caricature level has beenincreased in an inverse relationship to the caricature size. This willbe apparent by considering the smallest caricature image on the righthand side, and comparing the features thereof to the largest caricatureimage on the left hand side, and which has the least caricature levelapplied thereto. It will be seen that the smallest caricature imagewhich has had the greatest caricature level applied has more distortedfeatures than that of the largest caricature image but that the smallestcaricature image is still recognisable as being an image of the sameperson as that shown in the largest caricature image.

A second embodiment of the invention will now also be described. Thesecond embodiment is identical to that of the first embodiment save forthe manner in which the intended caricature image size is determined bythe control and interface program 36.

Within the second embodiment, the caricatured images generated by theembodiment of the invention are not merely displayed to a user, butinstead are used by a third party program such as, for example, the gameprogram 32. That is, imagine that the game program 32 is a programwherein facial images of different sizes must be displayed to the user.For example, the game program may be a virtual reality adventure gamesuch as is known in the art under the names of Quake™ or Doom™, oralternatively may be, for example, a sports simulation program such as afootball game, wherein the characters are known football players. Thegame program 32 is executed by the general purpose computer 1 under thecontrol of the user, and displays the faces of its characters on thedisplay screen of the general purpose computer 1 at different sizesdepending on the progression of the game. In order to aid recognition ofthe characters within the game, the facial features thereof may becaricatured using the techniques provided by the present invention, asfollows.

Imagine the game program 32 is being executed by the computer 1. Thevisual display provided by the game program 32 requires the facialfeatures of a character to be displayed on the screen at a certain size.The game program 32 stores a texture image of the characters face as afacial image. As part of its internal processing, the game program 32determines the size of the facial image of the character to bedisplayed. Then, prior to display the game program 32 interfaces withthe control and interface program 36, and passes the informationrelating to the intended image size, as well as the actual facial imageof the character to be displayed to the control and interface program36. The control and interface program 36 then uses this receivedinformation to determine the caricature level in the same manner aspreviously described in respect of the first embodiment, and then startsthe caricaturing program 34, passing the input data to the caricaturingprogram as previously described in the first embodiment. Thecaricaturing program 34 then produces a caricatured image using thereceived information, which is passed back to the control and interfaceprogram 36. The control and interface program 36 then passes back thegenerated caricature image to the game program 32, which uses thegenerated caricature image as a texture map for the face of thecharacter when the display generated by the game program is beingrendered for display on the screen of the general purpose computer 1.The result is that the face of the character as displayed on the screenis textured with the generated caricatured image hence resulting inincreased recognisability to a user of the game program 32.

Relating the above described second embodiment to the detaileddescription of the first embodiment as given previously, it will be seenthat the operation of the second embodiment is almost identical to theoperation of the first embodiment, except at the step 4.2 the controland interface program receives the input image from the game program 32,and similarly, at step 4.4 the control and interface program 36 receivesinformation determining the intended caricature image size from the gameprogram 32. The remaining steps of the process of the second embodimentare then identical to those previously described in respect of the firstembodiment, up until the image is generated. Once the caricature imageis generated within the second embodiment the control and interfaceprogram 36 passes the generated image back to the game program 32 fordisplay, rather than displaying the image directly to the user itself.The second embodiment of the invention therefore provides that the sizedependent caricaturing techniques provided by the invention may be usedby other applications, such as game programs or the like, by interfacingthe control and caricaturing programs provided by the embodiment withthose other applications.

Moreover, whilst in the second embodiment we have described thecaricaturing program 34 and the control and interface program 36 asbeing separate from the game program 32, this need not necessarily bethe case, and the caricaturing program 34 and control and interfaceprogram 36 may be fully incorporated within the game program 32 asmodules thereof.

Turning now to other modifications which may be made to either of theembodiments, with respect to the function which sets the caricaturelevel in dependence on the image size, the inventors have found that agenerally inverse function over at least a sub range of the possibleimage sizes as is shown in FIG. 5 is preferable, but the invention isnot limited to the use of such a function, and instead a function wherethe caricature level increases with image size may also be used.Moreover, the functions may be linear functions, non-linear functions,and also may be applied over the whole range of possible intended imagesizes rather than merely a sub range as shown in FIG. 5.

Unless the context clearly requires otherwise, throughout thedescription and the claims, the words “comprise”, “comprising” and thelike are to be construed in an inclusive as opposed to an exclusive orexhaustive sense; that is to say, in the sense of “including, but notlimited to”.

Moreover, for the avoidance of doubt, where reference has been given toa prior art document or disclosure, whose contents, whether as a wholeor in part thereof, are necessary for the understanding of the operationor implementation of any of the embodiments of the present invention bythe intended reader, being a man skilled in the art, then said contentsshould be taken as being incorporated herein by said reference thereto.

1. A method of generating a caricatured image, comprising the steps of:determining a caricature level value in dependence on an intended sizeof the caricatured image to be generated; and generating the caricaturedimage using the caricature level value thus determined.
 2. A methodaccording to claim 1, wherein the determining step further comprisesdetermining the caricature level value as a generally inverse functionof the intended size of the caricatured image to be generated.
 3. Amethod according to claim 2, wherein the function is applied over atleast a sub-range of the possible range of intended sizes of thecaricatured image to be generated.
 4. A computer program or suite ofcomputer programs arranged such that when executed by a computer systemit/they enable the computer system to operate according to claim
 1. 5. Acomputer readable storage medium storing the computer program or one ormore of the suite of computer programs according to claim
 4. 6. A systemfor generating caricatured images, the system comprising processingmeans arranged in use to: i) determine a caricature level value independence on an intended size of the caricatured image to be generated;and ii) generate the caricatured image using the caricature level valuethus determined.
 7. A system according to claim 6, wherein theprocessing means is further arranged in use to determine the caricaturelevel value as a generally inverse function of the intended size of thecaricatured image to be generated.
 8. A system according to claim 7,wherein the function is applied over at least a sub-range of thepossible range of intended sizes of the caricatured image to begenerated.